Modelling User Satisfaction and Student Learning in a Spoken Dialogue Tutoring System with Generic, Tutoring, and User Affect Parameters
نویسندگان
چکیده
We investigate using the PARADISE framework to develop predictive models of system performance in our spoken dialogue tutoring system. We represent performance with two metrics: user satisfaction and student learning. We train and test predictive models of these metrics in our tutoring system corpora. We predict user satisfaction with 2 parameter types: 1) system-generic, and 2) tutoringspecific. To predict student learning, we also use a third type: 3) user affect. Alhough generic parameters are useful predictors of user satisfaction in other PARADISE applications, overall our parameters produce less useful user satisfaction models in our system. However, generic and tutoring-specific parameters do produce useful models of student learning in our system. User affect parameters can increase the usefulness of these models.
منابع مشابه
Speech recognition performance and learning in spoken dialogue tutoring
Speech recognition errors have been shown to negatively correlate with user satisfaction in evaluations of task-oriented spoken dialogue systems. In the domain of tutorial dialogue systems, however, where the primary evaluation metric is student learning, there has been little investigation of whether speech recognition errors also negatively correlate with learning. In this paper we examine co...
متن کاملAdapting to Student Uncertainty Improves Tutoring Dialogues
This study shows that affect-adaptive computer tutoring can significantly improve performance on learning efficiency and user satisfaction. We compare two different student uncertainty adaptations which were designed, implemented and evaluated in a controlled experiment using four versions of a wizarded spoken dialogue tutoring system: two adaptive systems used in two experimental conditions (b...
متن کاملTODO: This is a placeholder. Final title will be filled later
Speech recognition errors have been shown to negatively correlate with user satisfaction in evaluations of task-oriented spoken dialogue systems. In the domain of tutorial dialogue systems, however, where the primary evaluation metric is student learning, there has been little investigation of whether speech recognition errors also negatively correlate with learning. In this paper we examine co...
متن کاملUncertainty Corpus: Resource to Study User Affect in Complex Spoken Dialogue Systems
We present a corpus of spoken dialogues between students and an adaptive Wizard-of-Oz tutoring system, in which student uncertainty was manually annotated in real-time. We detail the corpus contents, including speech files, transcripts, annotations, and log files, and we discuss possible future uses by the computational linguistics community as a novel resource for studying naturally occurring ...
متن کاملComparing the Utility of State Features in Spoken Dialogue Using Reinforcement Learning
Recent work in designing spoken dialogue systems has focused on using Reinforcement Learning to automatically learn the best action for a system to take at any point in the dialogue to maximize dialogue success. While policy development is very important, choosing the best features to model the user state is equally important since it impacts the actions a system should make. In this paper, we ...
متن کامل